A distributed file system enables a client to access and process data stored on a remote server as if it were on the client machine. In a distributed file system running on a high latency network (e.g., a wide area network), client machines may experience significant delay when attempting to retrieve data from a remote server especially when retrieving that data may require multiple network round trips (as described in further detail below). Distributed file systems may be centralized or decentralized. In a centralized serving environment, all data (e.g., directories and files) typically resides on the same machine as the metadata (e.g., data pertaining to the location of the files or directories and other such related information). A client may therefore access metadata and data from a single centralized server by navigating the server's file system. File systems are well known to those of ordinary skill in the art and further description thereof is omitted herein.
In a decentralized serving environment, on the other hand, the data and metadata may reside on different servers (e.g., the data, comprising the contents of files and directories, on data server(s) and metadata on metadata server(s)). In order for a client to retrieve data on a network comprising such a decentralized file system, the client first obtains metadata from a metadata server and uses this information to determine the location of the data. The client may then access the data from the appropriate data server location by navigating the data server's file system.
When a client retrieves a file (e.g., document File A) from a server, the client first sends a message to a metadata server, asking for information (metadata) pertaining to File A. The client then parses the metadata and retrieves the file from the appropriate data server. Thus, for example, if File A resides within Subdirectory Z, which resides within Subdirectory Y, which in turn resides within Directory X, the complete path for File A may look as follows: X/Y/Z/A.doc. A client attempting to retrieve File A therefore may first locate Directory X, then Subdirectory Y and Subdirectory Z, and finally retrieve File A. Although transparent to the user, file system navigation may differ, depending on whether the file system is centralized or distributed.
FIG. 1 illustrates a known method of retrieving data (“File A”) on a distributed network including a centralized file server. As illustrated, Network 100 comprises multiple sub-networks and includes Client 101 and Centralized Server 102. Centralized Server 102 includes Directory X, Subdirectory Y and Subdirectory Z, and Subdirectory Z includes File A.doc. Centralized Server 102 also includes all metadata corresponding to File A.doc (e.g., the locations of Directory X, Subdirectory Y, Subdirectory Z and File A.doc on Centralized Server 102). When Client 101 desires to access X/Y/Z/A.doc, Client 101 first obtains the metadata for Directory X, Subdirectory Y, Subdirectory Z, and File A.doc, and then uses all the retrieved metadata to access the contents of that file. An example of a distributed file system with a centralized server is UNIX-based Network File Server (Version 4, Internet Engineering Task Force, RFC 3010, hereafter “NFS V4”).
FIG. 2 illustrates a known method of retrieving File A.doc on a distributed, decentralized file system. As illustrated, Network 200 comprises multiple sub-networks and includes Client 201, Metadata Server 202 and Data Servers 203 and 204. Data Server 203 includes Directory X. The contents of all directories on Data Server 203, including Directory X, may be presented as a list of names and unique identifiers for all included subdirectories, and the metadata server on which the metadata for each of the included subdirectories can be found. In the illustrated example, Directory X may contain a single entry that indicates that Subdirectory Y's metadata is on Metadata Server 202. In turn, Subdirectory Y may contain only Subdirectory Z. Given the decentralized nature of the file system depicted in Network 200, however, Subdirectory Y may physically reside on Data Server 203 while Subdirectory Z may physically reside on Data Server 204.
When Client 201 desires to access X/Y/Z/A.doc, Client 201 may have to perform the following. First, Client 201 may obtain the metadata for Directory X from Metadata Server 202. Client 201 may then use this metadata to determine that the contents of Directory X are stored on Data Server 203. Client 201 may then retrieve the contents of Directory X and determine that Subdirectory Y exists and that Subdirectory Y's metadata is also stored on Metadata Server 202. Client 201 may again contact Metadata Server 202 to identify the location of the contents of Subdirectory Y. Once again, Client 201 may determine that Subdirectory Y's data is on Data Server 203. Client 201 may then contact Data Server 203 yet again to identify the contents of Subdirectory Y. Client 201 may determine that Subdirectory Z exists in Subdirectory Y and that the metadata for Subdirectory Z can be found on Metadata Server 202. Thereafter, Client 201 may contact Metadata Server 202 to identify where the contents of Subdirectory Z are stored. Upon determining that the contents of Subdirectory Z reside on Data Server 204, Client 201 may then contact Data Server 204 to get the contents of Subdirectory Z. Client 201 may then determine that File A.doc exists and that the file's metadata may be located on Metadata Server 202. Client 201 may then contact Metadata Server 202 to find the location of the contents of File A.doc and again determine that the contents of file A.doc are on Data Server 204. Finally, Client 201 may contact Data Server 204 to retrieve the contents of File A.doc.
As illustrated by the above figures, in either centralized or decentralized file systems, the client (Client 101 or Client 201) may need to send a set of several related messages (e.g., commands, queries, instructions, method invocations, etc.) in succession to a server to retrieve data. For example, when attempting to access File A in either scenario, the client may send a series of messages to the serving infrastructure to parse the directory structure and then to locate and access File A.
As illustrated in the above example, retrieval of File A from a data server on a distributed network may thus involve numerous messages over the network. In many distributed file systems, these messages must be sent one at a time because the output of one message may be the input to the next message. Thus, each message and/or query often requires a network roundtrip, from the client to the server and back. The network roundtrips are likely to be on a Wide Area Network (“WAN”), i.e., across networks with a high round-trip latency. These roundtrips are therefore likely to have a significant negative impact on performance.